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Multiple measures administered in repeated waves within a nonequivalent dependent variables quasi- 
experimental design were used to test the effects of a reform-oriented instructional method called En- 
hanced Anchored Instruction (EAI) on the math achievement of 128 middle school students, including 
students with learning disabilities (LD). EAI problems are presented in multimedia and hands-on for- 
mats, a potential benefit for students with low skills in both reading and math. Overall, students of all 
ability levels benefited from EAI with effect sizes (rp) ranging from .53 to .59. Results revealed that 
although students with LD scored lower on pretests, their learning trajectories matched those of stu- 
dents without LD. A maintenance test administered several weeks after instruction showed that stu- 
dents with LD retained what they had learned. Implications for instruction and suggestions for future 
research are provided. 


Government and professional groups have urged educators 
to help all students acquire mathematical preparedness for 
post-secondary education and employment (e.g., Standards for 
Technological Literacy: Content for the Study of Technology , 
International Technology Education Association, 2000; Goals 
2000: Educate America Act, U.S. Department of Education, 
1994; What Work Requires of Schools: A SCANS Report for 
America 2000 , U.S. Department of Labor, 1991). The most re- 
cent of these initiatives, the No Child Left Behind Act of 2001 
(NCLB), outlines a national initiative for improving elemen- 
tary and secondary education tied to high-stakes assessments. 
In response to these pressures, the National Council of Teach- 
ers of Mathematics (NCTM, 2000) has called for curricular 
reform that emphasizes more problem-based learning. Accord- 
ing to the NCTM, these problems should develop the skills 
and concepts of middle school students in (a) working flexi- 
bly with whole numbers, fractions, and decimals; (b) construct- 
ing and interpreting scale drawings; (c) converting units of 
measure; and (d) interpreting tables and graphs. 

Recent test scores show that these reforms may be pay- 
ing off. Results from the National Assessment of Educational 
Progress (Perie, Grigg, & Dion, 2005) indicated that eighth 
graders scored higher in 2005 than in any previous year since 
the test was administered. However, this good news was ac- 
companied by less positive findings showing more than one 
quarter of students without disabilities (28%) and more than 
two thirds of students with disabilities (69%) still scoring below 
Basic performance levels. Basic means students “should com- 


plete problems correctly with the help of structural prompts 
such as diagrams, charts, and graphs” and include “the ap- 
propriate use of strategies and technological tools to under- 
stand fundamental algebraic and informal geometric concepts 
in problem solving” (p. 20). Thus, the new standards call for 
a range of skills beyond procedural competency. 

Higher expectations coupled with the sluggish math per- 
formance of students with disabilities have led some special 
educators (e.g., Jones, Wilson, & Bhojwani, 1997; Woodward, 
2004; Woodward & Baxter, 1997; Woodward & Montague, 
2002) to question whether traditional instructional methods 
for students with learning disabilities (LD) are appropriate 
and adequate. Central to this issue is how and to what extent 
the teaching practices used in special education — which have 
leaned toward behaviorist principles — can and should be modi- 
fied to align with current reform-oriented practices in general 
education promoting more constructivist methods. 

Shifting to a more constructivist approach will not be 
easy for several reasons. First, special educators (e.g., Vaughn, 
Klingner, & Hughes, 2000) have cautioned against embrac- 
ing the reform-oriented methods for less capable students with- 
out an adequate research foundation because (a) students who 
have low math skills may not be able to solve more difficult 
types of math problems advocated by NCTM and (b) having 
such students spend time on problems they are not able to solve 
may limit the time available for basic skills. Various meta- 
analyses of special education research have shown the benefits 
of drill and repetition for many students with LD (Swanson, 
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2001; Swanson & Hoskyn, 1998). These methods carefully 
sequence content so task difficulty does not overload students’ 
working memory (e.g., Carnine, 1998; Swanson & Deshler, 
2003). Abandoning proven approaches at this time may be un- 
wise, especially when general education is reevaluating their 
instruction of procedural skills (Star, 2005). 

A second important question concerns how to individu- 
alize instruction for students with LD who are certain to face 
more complex content in general education math classrooms. 
In the past, most students with LD received their math in- 
struction in small-group settings where teachers tailored spe- 
cialized instruction to each student’s learning needs. As more 
students with LD are included in general education classrooms, 
the question of whether and how they will get the support in 
regular education classes is still largely unanswered (Baker & 
Zigmond, 1990). Research suggests that pullout settings do 
not produce satisfactory results in school achievement over the 
long term (Fuchs & Fuchs, 1995; Rea, McLaughlin, Walther- 
Thomas, 2002), but few studies have identified practices that 
promote effective math instruction for students with LD in in- 
clusive, reform-oriented settings. 

The importance of these issues (i.e., curriculum, indi- 
vidualization) increases in complexity as students with LD 
transition to secondary school settings. Deshler et al. (2001) 
identified the struggles students encounter with higher and 
more focused academic expectations. Their graphic contrast- 
ing the upward learning trajectory of typical achievers and the 
learning plateau of students with LD is particularly informa- 
tive. Although a variety of approaches has improved the aca- 
demic performance of adolescents with LD in a number of 
important areas, such as algebra (Hutchinson, 1993; Maccini & 
Hughes, 2000; Witzel, Mercer, & Miller, 2003), equivalent frac- 
tions (Butler, Miller, Crehan, Babbitt, & Pierce, 2003), and 
problem solving (Jitendra, DiPipi, & Perron-Jones, 2002; Ji- 
tendra, Hoff, & Beck, 1999; Montague, 1997, 1998; Montague 
& Bos, 1986), more research is needed to show how these ap- 
proaches can be used alongside of or embedded in general ed- 
ucation curricula. 

One instructional method that may have the potential for 
helping students with diverse abilities is called Enhanced An- 
chored Instruction (EAI; Bottge, 2001). Based on the concept 
of anchored instruction (AI; Cognition and Technology Group 
at Vanderbilt, 1990, 1997; Hickey, Moore, & Pellegrino, 2001), 
students first solve a problem in a multimedia format and then 
apply what they leam in related hands-on problems (e.g., 
building skateboard ramps, designing and manufacturing hov- 
ercrafts). A mix of experimental and quasi-experimental stud- 
ies in inclusive and pull-out settings has yielded effect sizes 
(/• 2 ) from .31 to .66 on problem-solving tests and from .14 to 
.38 on transfer tasks (Bottge, 1999; Bottge, Heinrichs, Chan, 
& Serlin, 2001; Bottge, Heinrichs, Mehta, & Hung, 2002). 
The focal point of these studies was on developing students’ 
understanding of complex, multistep problems that required 
knowledge of fractions, linear measurement, and representa- 
tional skills. One important advantage of EAI is its ability to 


directly immerse students in problem contexts, thus helping 
to eliminate the comprehension difficulties students with low 
skills in both math and reading often experience with com- 
plex text-based problems (Fuchs & Fuchs, 2002; Lesh & 
Kelly, 2000). 

Although the EAI research has shown positive benefits 
for many students, the approach has its disadvantages. As 
Woodward (2004) pointed out, the difficulty level of anchored 
problems is high, which may at times overload the working 
memory of students with LD. In earlier studies with a limited 
number of scaffolds in place to reduce the cognitive load, 
some students with LD became frustrated by the complexity 
of EAI problems. A second issue relates to the lack of speci- 
ficity in descriptions of the concepts and skills in the EAI 
problems. Prior reports of EAI have included overviews of 
learning objectives, but they did not provide specifics about 
what the assessments measured. Finally, the EAI research has 
highlighted the need to more adequately train teachers who 
use EAI, especially on how to organize and manage hetero- 
geneous groups. In EAI settings, students with LD work with 
their classmates in small problem-solving groups, and teach- 
ers must be vigilant to ensure that students work cooperatively. 
In previous studies, group processes have at times broken 
down, and students with LD simply copied the work of more 
able students. 

The goal of this study was to measure the effects of two 
EAI problems on the math performance of middle school stu- 
dents across a range of ability levels after improvements had 
been made in curricular design, instructional methods, and 
teacher training. Specifically, we assessed whether and to what 
extent the achievement and learning trajectories of students 
with and without LD widened, narrowed, or stayed the same 
as a result of working on two problems: 

1 . an EAI problem designed to teach students to 
calculate rates, interpret tables and construct 
graphs, and construct line of best fit for the 
purpose of making predictions based on rela- 
tionships between variables 

2. an EAI problem designed to teach students to 
interpret schematic plans, measure lengths in 
feet and inches, and compute combinations 
with whole numbers and fractions for the pur- 
pose of estimating the total cost of a building 
project. 

Method 

Participants 

A total of 128 seventh-grade students in six intact middle school 
math classrooms participated in the 7-month study. The mid- 
dle school is located in a small town in the upper Midwest. 
Two math teachers (MT1, MT2) each taught three 90-min 
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blocks per day. Table 1 shows descriptive information for stu- 
dents in the six classes. MT1 taught an inclusive class, a pre- 
algebra class, and a class of students considered neither 
particularly high nor low achievers (i.e., typical). The inclu- 
sive class consisted of 13 students with disabilities, most with 
LD (hereafter referred to as LD), and a group of 13 students 
without disabilities who were considered to be typical achiev- 
ers. The students with LD were placed in one class to allow 
the special education teacher or her assistant to provide extra 
help. The school had identified students for the pre-algebra 
class based on their math performance in sixth grade. Students 
had to have maintained an A average, completed 95% of their 
homework, attained at least 85% on a teacher-developed math 
test, and scored at or above the 90th percentile on the state 
math test. The students in MTl’s other class and in all three 
of MT2’s classes were considered by the school to be typical 
achievers because their skills were not high enough to be rec- 
ommended for the pre-algebra class and not low enough to be 
referred for remedial help. 

Table 2 shows demographic information of the students 
with LD in the inclusive class. Individualized Education Pro- 
grams (IEPs) revealed that these students were receiving spe- 
cial education services an average of 625 min a week (range = 
300-920). Twelve of the students were receiving special ed- 
ucation services for LD, and the majority of them were re- 
ceiving services in more than one content area. Two students 
with disabilities were considered at risk in math but had not 
yet been referred for LD services. According to the Wiscon- 
sin Department of Public Instruction (2006), students with LD 
have a significant discrepancy equal to or greater than 1.75 


standard errors of the estimate below expected achievement 
based on individually administered, standardized achieve- 
ment and ability tests. All of these students were male and 
Caucasian except one, who was an African American student. 

Mean national percentile ranks (NP) on the TerraNova 
Comprehensive Test of Basic Skills (CTB/McGraw-Hill, 1997), 
which all students took the previous spring as part of statewide 
testing, showed that the students with disabilities in the in- 
clusive class were achieving below average in math (NP =16) 
and reading (NP = 19). The Iowa Tests of Basic Skills (ITBS; 
University of Iowa, 2001), which was administered immedi- 
ately prior to the study, confirmed their low ability in com- 
putation (NP =15) and in problem solving (NP =14). 

Based on the ITBS results, MTl’s three classes differed 
in computation, F( 2, 68) = 67.03, p = .00, rp = .66, and in 
problem solving, F( 2, 68) = 38.57, p = .00, rp = .53. Pre- 
algebra students outscored students in the typical classes in 
computation, t = 5.62, p = .00, and in problem solving, t = 
3.87, p = .00; and students in the typical class scored higher 
than did students in the inclusive class both in computation, 
t = 5.45, p = .00, and in problem solving, t = 3.90, p = .00. 
MT2’s typical classes did not differ in computation, F( 2, 51) = 
1.36, p = .26, rp = .05, or in problem solving, F( 2, 51) = .85, 
p = .43, rp = .03, nor were there differences between the four 
typical classes (one of MT1 and three of MT2) in computa- 
tion, F( 3, 71) = 2.15, p = .10, rp = .08, or in problem solv- 
ing, F( 3, 73) = .76, p = .50, r) 2 = .03. 

The two math teachers planned and were primarily re- 
sponsible for instruction in the six classes. The special edu- 
cation teacher or the special education assistant was on hand 


TABLE 1. Description of Students in the Inclusive, Pre-Algebra, and Typical Math Classes 



Math Teacher 1 (MT1) 

Math Teacher 2 (MT2) 

Inclusive 

Pre-Algebra 

Typical 

Typical 

Typical 

Typical 

Total 

26 

27 

21 

16 

17 

21 

Gender 







Boys 

22 

8 

8 

6 

5 

11 

Girls 

4 

19 

13 

10 

12 

10 

Special education IEP 

13 

0 

0 

0 

0 

0 



ITBS pretest scores 




Computation 







M 

202 

259 

231 

216 

228 

220 

SD 

12.55 

17.37 

17.63 

19.22 

26.22 

12.19 

(NP) 

(17) 

(80) 

(51) 

(32) 

(47) 

(37) 

Problem solving & data interpretation 







M 

208 

280 

249 

252 

244 

238 

SD 

37.03 

18.41 

27.65 

28.36 

31.68 

34.60 

(NP) 

(28) 

(86) 

(65) 

(67) 

(61) 

(56) 


Note. IEP = Individualized Education Program; ITBS = Iowa Tests of Basic Skills, Level 12, Form A (University of Iowa, 2001); NP = national percentile. 
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TABLE 2. 

Description of Students With Disabilities 

in the Inclusive Class 











ITBS 



TerraNova CTBS 




Disability 

label(s) 

(Service 

areas) 

Minutes 
per week 
in special 
education 

Computation 

Problem solving 
& data 
interpretation 

Math 

Reading 

Student 

birth 

SS 

NP 

SS 

NP 

SS 

NP 

SS 

NP 

Alex 

5/9/90 

LD/ADD 

(M,R,L) 

600 

200 

15 

131 

1 

588 

5 

593 

6 

Mark 

3/3/90 

LD/EBD 

(M,R,L) 

500 

191 

10 

173 

5 

619 

15 

613 

12 

John 

4/8/90 

SL/LD 

(L) 

450 

200 

15 

213 

33 

641 

30 

635 

26 

Ted 

3/31/90 

LD 

(M,L) 

750 

182 

5 

204 

24 

613 

13 

554 

2 

Rob 

6/14/90 

LD/ADHD 

(M,R,L) 

540 

209 

24 

200 

20 

623 

17 

629 

22 

Jim 

3/27/90 

LD/SL 

(L) 

665 

191 

10 

185 

10 

n.a. 

n.a. 

n.a. 

n.a. 

Les 

3/11/90 

LD 

(M,R,L) 

880 

204 

19 

185 

10 

551 

i 

578 

4 

Aaron 

9/4/89 

LD 

(M,R,L) 

800 

191 

10 

173 

5 

551 

i 

529 

1 

Brian 

1 1/29/89 

LD/EBD 

(M) 

300 

196 

12 

200 

20 

592 

6 

543 

1 

Rick 

1 1/25/89 

LD/ADD 

(M,R) 

480 

217 

33 

161 

1 

570 

2 

602 

8 

Willy 

10/17/89 

LD 

(M,R,L) 

920 

221 

38 

185 

10 

604 

9 

597 

7 

Ed 

5/26/89 

LD 

(M,R) 

n.a. 

186 

7 

248 

64 

595 

7 

550 

1 

Phil 

12/27/89 

ADD 

n.a. 

191 

10 

233 

51 

n.a. 

n.a. 

n.a. 

n.a. 


Note. LD = learning disability; ADD = attention-deficit disorder; EBD = emotional and behavioral disorder; SL = speech and language disability; ADHD = attention-deficit/ 
hyperactivity disorder; M = math; R = reading; L = written language; ITBS = Iowa Tests of Basic Skills, Level 12, Form A (University of Iowa, 2001); CTBS = Comprehensive 
Test of Basic Skills (CTB/McGraw-Hill, 1997); SS = scale score; NP = national percentile; n.a. = not applicable. 


to help students with LD in the inclusive class, but they pro- 
vided no direct, whole-group instruction. MT1 was chair- 
person of the math department, had a master’s degree in 
middle-level education, was beginning his 13th year of teach- 
ing, and had incorporated EAI into his math classes the past 
three years. MT2 had a bachelor’s degree, was in her second 
year of teaching, and had no previous experience teaching 
with EAI. The special education teacher was in her 15th year 
of teaching, had a master’s degree in general education grades 
1 through 8, and was licensed to teach special education in 
grades kindergarten through 12. Her assistant did not have a 
college degree, but she had several years of experience work- 
ing with students with disabilities. 


Instructional Methods and Materials 

Instruction involved two EAI problems that were integrated 
into the math curriculum during one academic school year. 
The teachers taught Kim’s Komet for 13 days in October and 
Fraction of the Cost for 1 1 days in March. Between these times, 
the math teachers followed their prescribed curriculum, Con- 
nected Mathematics (Dale Seymour Publications, 2004). 

MT1 and MT2 assigned two to four students to problem- 
solving groups based on their ability to work together. In the 
inclusive class, one student with LD worked with two or three 
students without disabilities. At the beginning of each class 
period, the teachers led students in a 10-min warm-up activ- 
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ity to review concepts they had worked on the previous day 
and to introduce new material. For the remainder of class, each 
group worked on a laptop computer, which contained the 
learning tools (i.e., scaffolds) students could use to help solve 
each of the subproblems. Teachers circulated from group to 
group, answering questions and posing new ones. After stu- 
dents solved the media-based anchor, they worked on the 
hands-on applications. 

Kim’s Komet Instruction. Kim’s Komet is one episode 
in a series of video-based anchors called The New Adventures 
of Jasper Woodbury (Learning Technology Center at Vander- 
bilt University, 1997). As stated in the teacher manual, the pur- 
pose of Kim’s Komet is to help students develop their informal 
understanding of pre-algebraic concepts, such as linear func- 
tion, line of best fit, variables, rate of change (slope), and reli- 
ability and measurement error. Foundation skills needed to 
solve this problem include computation with whole numbers 
and decimals. 

The video anchor involves two girls who compete in 
pentathlon events. The first challenge asks students to identify 
the three fastest qualifiers in three regional races, where the 
times and distances are known but the distances vary. Students 
learn, for example, whether a car that travels 15 ft in 0.9 s is 
faster or slower than a car that travels 20 ft in 1.3 s. The sec- 
ond challenge asks students to construct the “line of best fit” 
on their graph to predict the speed of cars at the end of a 
straightaway after they have been released from various heights 
on the ramp. The video gives students the opportunity to clock 
the speed of Kim’s car from several heights. Eventually stu- 
dents realize that they should begin timing their cars on the 
straightaway, where the car’s speed is relatively constant, rather 
than on the ramp, where the car is accelerating. The video 
helps students understand the concept of reliability of mea- 
surement, because students measure the speeds several times. 

After students solved the problems in the video anchor, 
they participated in their own pentathlon competition by 
building and testing cars they made. The technology educa- 
tion teacher at the school built the ramp, straightaway track, 
and pentathlon stunts shown in the video anchor. At the be- 
ginning and end of the straightaway, an infrared detector mea- 
sured the time it took for the students’ cars to travel from one 
end of the straightaway to the other. Using these times, stu- 
dents made their own graphs showing the speeds of their cars 
for each release point on the ramp. Their graphs helped them 
predict where on the ramp to release their cars to achieve the 
speeds necessary to successfully navigate the stunts at the end 
of the straightaway. 

Standard Instruction. After teaching Kim’s Komet and 
before introducing Fraction of the Cost, the math teachers 
taught units on concepts related to geometry and proportional 
reasoning using their prescribed curriculum, Connected Math- 


ematics (Dale Seymour Publications, 2004). The students com- 
pleted several projects, such as figuring out the cost of an ad- 
vertisement based on the proportion of the full page it covered. 
The teachers organized their instruction during this phase in 
ways that were similar to EAI. That is, they reviewed previ- 
ous work, taught new concepts explicitly, and then provided 
time for students to solve related problems. 

Fraction of the Cost Instruction. The authors devel- 
oped the 8-min video-based anchor called Fraction of the Cost, 
which stars three students from a local middle school. Avail- 
able in Spanish and English, the video was filmed at a local 
skateboarding store, a garage, and the backyard of a local home. 
The video opens in a skateboard store and rink, where the stu- 
dents are shown discussing how they can afford to buy mate- 
rials for building a skateboard ramp. To solve the problem, 
students need to (a) calculate the percent of money in a sav- 
ings account and sales tax on a purchase, (b) read a tape mea- 
sure, (c) convert feet to inches, (d) decipher building plans, 
(e) construct a table of materials, (f) compute whole numbers 
and mixed fractions, (g) estimate and compute combinations, 
and (h) calculate total cost. 

Instruction proceeded in much the same way as with 
Kim’s Komet. Teachers showed the 8-min problem-solving 
anchor Fraction of the Cost without interruption the first day 
and asked students to describe the problems associated with 
it. Students then worked in small groups on their problem- 
solving packets. To figure out the most economical use of two- 
by-four (2" x 4") lumber in Fraction of the Cost, students first 
needed to identify the lengths of boards shown in the plan. 
Then they converted these dimensions from feet and inches to 
inches only and calculated several combinations of lengths to 
maximize their use of the wood available. Once they com- 
puted what additional wood was needed, they consulted the 
store advertisement to decide what to buy. Their final task was 
to make a materials list to show the total cost of the project. 

To help solidify students’ understanding, the math teach- 
ers often led students in brief practice sessions the first 10 min 
of class. For example, one day the teachers projected on the 
projector screen four narrow rectangles of various lengths that 
represented two-by-fours and asked students to respond to a 
series of questions about converting feet and inches to inches 
only and the most economical way of cutting the boards the 
students in the video found in the garage: 

Teacher: Okay, eyes up here. [Points to a board 
labeled 2 feet, 5 14 inches.] What is 
this in inches? 

Cindy: 2 feet, 514 inches ... 12 inches x 2 = 

24 inches ... 24 inches + 514 inches = 

2914 inches. 

Teacher: Good. [Points to a board labeled 
5 feet, 1 114 inches.] 

Greg: 5 feet, 1114 inches ... 5 x 12 inches = 
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Teacher: 

All Students: 
Teacher: 

Brian: 


Teacher: 
All Students: 
Teacher: 


All Students: 
Teacher: 

All Students: 

Teacher: 
All Students: 
Teacher: 

All Students: 
Teacher: 

All Students: 


60 inches ... 60 inches +11% inches = 
71% inches. 

Can I cut this piece (29%") out of a 
board 2 feet, 5% inches long? 

No! 

Can I cut 79 inches from a board that is 
8 feet long? Why or why not? 

8 feet x 12 inches is 96 inches, and 96 
inches is longer than 79 inches ... so the 
answer is yes! 

How much is left over? 

17 inches. 

Are there any of these boards [pointing 
to schematic plan] that I can cut from 
what’s left, 17 inches? 

16% inches. 

How much wood do I lose each time 1 
cut? 

% inch [referring to the kerf of the saw 
blade]. 

How many times did I cut? 

Twice. 

So how much wood did I lose from cut- 
ting the board twice? 

2 % inches, or 14 inch. 

Let’s take a vote. Should 1 cut the 8-foot 
board into 16% inches and 79 inches? 
Yes! 


After students solved the problems posed in Fraction of 
the Cost, they worked on solving a related problem called the 
Hovercraft Challenge, in which students had to plan and con- 
struct out of PVC pipe a “rollover cage” for a hovercraft. The 
teacher divided the class into groups of three students, and 
each group planned how they could make the cage in the most 
economical way. Once the teacher approved the plans, stu- 
dents worked on measuring, cutting, and assembling. When 
the cages were complete, they lifted them onto a 4' x 4' ply- 
wood platform. A leaf blower inserted into a hole in the ply- 
wood powered the hovercraft. On the last day of the project, 
students rode on their hovercrafts in relay races up and down 
the halls of the school. 


Instrumentation 

Two tests, the Kim's Komet Challenge (KKC) and the Frac- 
tion of the Cost Challenge ( FCC ), assessed minimally over- 
lapping sets of discrete math concepts embedded in the two 
EAI problems. Constructed-response items measured knowl- 
edge of mathematics aligned to NCTM (2000) standards rec- 
ommended for students in Grades 6-8 (i.e.. Numbers and 
Operations, Measurement, Problem Solving, Communication, 
and Representation). Each test went through cycles of refine- 
ment based on student performances in previous research 
(Bottge, Heinrichs, Mehta, & Hung, 2002; Bottge et ah, 2004; 


Bottge, Heinrichs, Chan, & Serlin, 2001; Hung, 2005) and on 
suggestions from math and assessment specialists (i.e., math 
teachers, math researchers, test consultants). Sets of problems 
were weighted according to their complexity and the contri- 
bution they were expected to make in solving the overarching 
problem. Within each set, items were awarded full or partial 
credit, which made it possible to analyze student work at both 
the item and concept levels (Lester & Kroll, 1990; Shafer & 
Romberg, 1999). Concurrent validity correlation coefficients 
based on pretest scores of the Iowa Tests of Basic Skills (ITBS) 
Problem Solving and Data Interpretation Subtest (University 
of Iowa, 2001) were .59 for the FCC and .52 for the KKC. 
Both correlations were significant and appear acceptable given 
that the range of mathematics concepts sampled by FCC and 
KKC was more restricted than that sampled on the ITBS. In- 
ternal consistency (Cronbach’s coefficient alpha) of KKC and 
FCC were .92 and .80, respectively. The research staff cal- 
culated interrater reliabilities on 20% of the test protocols 
randomly selected from each of the scheduled test adminis- 
trations and were computed by dividing the number of agree- 
ments by the total number of agreements and disagreements 
and multiplying by 100 (Sulzer-Azaroff & Mayer, 1977). In- 
terrater reliability was 95% (range = 93-98%) for the Kim’s 
Komet test and 94% (range = 90-97%) for the Fraction of the 
Cost test. 

Kim’s Komet Challenge. The 36-point Kim's Komet 
Challenge assessed students’ ability to estimate and compute 
rates when times and distances are known. Students were as- 
sessed on their ability to interpret data in tables, to construct 
graphs, and to make predictions based on relationships be- 
tween variables. These concepts and skills match closely 
those identified in the NCTM (2000) Algebra Standards for 
Grades 6-8. 

Figure 1 shows three sets of test items (Items 5-7), which 
together account for 28 of the 36 total points. Student work is 
included to show correct responses. In Item 5, students use 
the information in the table to calculate the speeds of six cars, 
in feet per second, when distance and time are provided. The 
figure above the table is intended to depict the ramp and 
straightaway as they are shown in the instructional anchors 
(i.e., video-based and hands-on). After calculating each of the 
speeds, students indicate who in each town has the slowest 
and fastest cars and which car is fastest overall. Students can 
earn a total of 16 points if they calculate all speeds correctly 
or partial credit if they answer only some of the items cor- 
rectly. 

In Item 6, students calculate how fast cars are traveling 
at the end of the straightaway after they have been released 
from several heights on a ramp. This problem is difficult be- 
cause students must first understand why they should not 
begin timing the cars at the release point on the ramp (i.e., 
heights of 44", 60", 108”). During instruction, students learn 
that cars accelerate on the down ramp and therefore accurate 
speeds cannot be computed. Instead, they must use the time 


Item 5 


Two groups of high school students were racing their cars down a ramp. The ramp 
is show below. The car tracks were different lengths. 

START 



a. Calculate the speeds of the Lodi and Sun Prairie cars. Write the speeds in the 


tables below. 


LODI STUDENTS 


Your answer 
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LENGTH OF TRACK 

TIME 

SPEED 
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b. Who has the fastest car 
in Lodi? 


c. Who has the fastest car in 
Sun Prairie? 


d. In Lodi and Sun Prairie, who has 
the fastest car? 


e. In Lodi and Sun Prairie, who has 
the slowest car? 


C^rb_'> 

Your answer 

Your answer 

C'S'i'i’’’ 


>- 

Your answer 




> 


Scoring Protocol 


0. 

Speed 

Points , 

Ann 

6.25,6.3,6.2 


Alary 

5 

2/ 

Rich 

3.6, 3.5, 3.57 

zv. 

Tom 

6 


Jan 

7.5 

tl 

Dave 

4.7. 4.6. 4.68 

2V, 

b. Ann 



c. Jon 



d. Jon 



e. Rich 


IT 

Total points X 6/16 


Item 6 


Calculate the speed of a car (feet per second) dropped from each height on the ramp, 
a. at 44 inches 



W*rkAm 


• 2.5 seconds - 




ut 


r r r r , r 

O' 5' 10' 15' 20’ '» i 


Your answer 


b. at 60 inches 




2.2 seconds 


k— - 13 seconds — 

£ g r r 

0‘ 5* 10’ 15’ 


pr 

20 ’ 


Work Arcs 

15.1 


Your answer 



Scoring Protocol 


Work Area 


Partial 

Pull 

@44“ 

20> 2.5 sec. = 

8 ft/sec 

1 

zJ 

@60“ 

20 + 1.9 sec. = 

10.5. 10.52. 10.53 

1 

2^ 

@108 

2ff * 1.3 sec s 

15.4. 15.3, 15.38 

1 

2 / 


Totol points 6 /6 


Item 7 

Make a graph to show the speed of the car for each height on the ramp. 
Please be sure to label the X axis and the Y axis. 






Scoring Protocol 


X-Axis (correct label and 
correct scale) Speed or Rate 
inclusive of 16 OR Distance or 
Height inclusive of 108 

Partial 

Full 

1 

2 l/ 

y-Axis (correct label and 
correct scale) Speed or Rate 
inclusive of 16 OR Distance or 
Height inclusive of 108 

1 

2 / 

Line graph plotting 3 points: 
(44. 8) (60, 10.5) (108, 15.4) 

1 

2 / 

Total points 


fi/6 


FIGURE 1. Representative items and student work on Kim’s Komet Challenge (KKC). 
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it takes for the cars to travel from the beginning to the end of 
the straightaway, where speeds are more constant. Students 
calculate the speeds of their cars by dividing the length of the 
straightaway by the time taken to travel the straightaway. Stu- 
dents can earn partial credit for showing correct procedures. 

Item 7 asks students to graph the data from Item 6, label 
both axes, and draw a line of best fit. During instruction, stu- 
dents learn that this line shows the relationship between the 
height of the release point on the ramp and the speeds the cars 
will be traveling as they approach the end of the straightaway. 
The graph helps students pinpoint how fast a car will be trav- 
eling at the end of the straightaway for every release point 
(i.e., height) on the ramp. The scoring protocol shows that stu- 
dents can receive full credit for correctly graphing the data or 
partial credit if some of the information is correct. Students 
can earn partial or full credit in six major categories with scale 
weights based on more than 20 procedures. 

Fraction of the Cost Challenge. The 37-point Frac- 
tion of the Cost Challenge assessed students’ ability to inter- 
pret a three-dimensional schematic plan, measure lengths of 
building materials in feet and inches, estimate and compute 
combinations using whole numbers and fractions, interpret 
and record data in tables, and calculate total cost including 
sales tax. The test emphasized real-world knowledge such as 
reading a bank account statement, comparing treated and un- 
treated lumber, selecting appropriate lengths and quantities of 
screws for specific applications, and developing a materials 
list with a cost estimate. Like the KKC, the procedures in the 
FCC were weighted according to difficulty and their contri- 
bution to the overall problem. 

Figure 2 shows two sets of items, scoring protocols, 
and the information students use to answer each problem. In 
Item 3, students use the rulers to measure the lengths of boards 
that appear in the anchor problem. Students are to record the 
lengths to the nearest Vs-inch in either feet and inches or in 
inches alone. Item 4 is more difficult because it requires stu- 
dents to show the most economical way to cut the wood shown 
in Item 3 for building the skateboard ramp. To solve this prob- 
lem, students must first be able to “read” the schematic plan, 
identify the number of boards of each length, and then figure 
out the most economical way to use the available wood. Then 
they use these combinations to indicate where to saw the 
boards. Like the KKC, the FCC requires students to integrate 
and manage several problem sets. 

Research Design 

Multiple measures administered in repeated waves within a 
nonequivalent dependent variables design assessed the effects 
of EAI on the achievement of students by ability and disabil- 
ity status. A unique feature of this design is that all students 
have access to the instructional interventions. At the most 
basic level, it involves a single group of students tested on two 
scales that are conceptually similar, but only one of which is 
expected to change because of the treatment. Adding multi- 


ple, repeated measures and predicting achievement patterns 
(i.e., pattern matching), as was done in this study, minimizes 
plausible threats to internal validity such as maturation, his- 
tory, and testing (Cook & Campbell, 1979; Shadish, Cook, & 
Campbell, 2002). Figure 3 shows the schedule of instruction 
and testing represented in standard design notation. 

Prior to instruction, students in all six classes were ad- 
ministered both problem-solving tests, KKC (0 [A ) and FCC 
(0 1B ). After teachers taught Kim’s Komet (X A ), students took 
the KKC (0 2A ), which served as a posttest for the concepts 
they were just taught, and the FCC (0 2B ), which assessed math 
concepts not emphasized during Kim’s Komet. For the next 
14 weeks, the teachers taught with their usual math curricu- 
lum. In the third wave of testing, all students took the KKC 
(0 3A ), which assessed students ’ retention of concepts they had 
last worked on several weeks before, and the FCC (0 3B ), 
which served as the pretest for Fraction of the Cost instruc- 
tion. In the last test wave, students took the FCC (0 4B ), which 
measured what students learned from Fraction of the Cost. 

A unique feature of this design is that the treatment is 
planned to affect only one of the measures at a time, and thus 
each outcome measure serves as a control for the other. Prior 
to conducting the study, we predicted that students would 
show overall improvement from test waves 0 [A to 0 2A while 
maintaining stable achievement from 0 1B to 0 2B and that, fur- 
thermore, the achievement pattern at 0 1A to 0 2A would be 
repeated at 0 3B to 0 4B . We also predicted that this pattern 
would align closely with ability and disability status. That is, 
students in the pre-algebra class would score higher than stu- 
dents in the typical math classes; students in the typical math 
classes would score higher than students in the inclusive class; 
and, students with LD would score lower than all of the other 
groups. 

Fidelity of Implementation 

Direct observation and video were used to monitor instruction 
in the math teachers’ classrooms. Detailed daily lesson plans 
developed especially for use with the EAI problems and fre- 
quent checks of lesson implementation helped ensure that the 
instruction proceeded as planned. Classroom observers 
recorded whether teachers stated the objectives for the day, 
led students in warm-up review exercises, reviewed concepts 
from previous classes, introduced new concepts related to the 
EAI problems, allowed time for students in their small groups 
to discuss possible solutions, and attended to the needs of in- 
dividual students. A primary and second observer took field 
notes 100% (24 days) and 67% (16 days) of the time, respec- 
tively, in the inclusive classroom. A primary and secondary 
observer also took field notes 7 1 % ( 1 7 days) and 1 3 % (3 days) 
of the time, respectively, in the MT2 classes. 

In addition to the observational data, the teaching prac- 
tices of MT1 and the special education staff in the inclusive 
classroom were chronicled with two cameras. One stationary 
camera was positioned in the back of the room facing the 
teacher. The wide-angle feature of the camera captured almost 
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Item 3 


What are the lengths of the wood in the garage? 




Scoring Protocol 



Feet 

Inches 

•Points 



5‘10F 

70 F 

0,1,2 correct^ 0 
3. 4 corrects 2 
All correct s 4 

•Select one 
column (feet Oft 
inches) to 


4‘II-P 

59 r 


T8i" 

92 Y 


5*U“ 

61 V 


award points. 


2-4 -r 

28 V 



Total Wood in Garaae section =4/4 



Item 4 


Label the wood that you found in the garage (drawn below) and show how 
you would use the wood in your skateboard ramp. Make sure that you show ~ 
how much wood is left in each piece. 


Scoring Protocol 

Feet 

Inches 

label 

Use 

5'10i” 

70 4” 

Correctly labels 
3 to 5 boards in 
feet or inches 
* 1 pt. 

1 pt. For each board 
cut in an economical 
way. 

1 2 3 4 5 

4'!li' 

59 4” 

78-f” 

92 Y 

5'1 V 

61 Y 



28 V 

Total aaraae wood section =6/6 



Information from the Video 



FIGURE 2. Representative items and student work on Fraction of the Cost Challenge (FCC). 
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Test 

Groups 





• Distance/Rate/Time 

• Graph/Predictions 

KKC 

Inclusive 

O-IA 

x A 

^2A 

O3A 

• Linear Function 

• Computation Whole Numbers 

Typical 

O-IA 

^A 

0 2 A 

s 

0 ( 

and Decimals 


Pre-Algebra 

O-iA 

X 

> 

^2A 

o 3A 


Linear Measurement 
Combinations 

FCC 
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OlB 

O2B 

O3B 

x B 

04B 

Visualization/Representation 
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O-IB 

0 2 B 

O3B 

x B 

04B 

Computation: 

Whole Numbers, Fractions 


Pre-Algebra 

O-IB 

0 2 B 

O3B 

m 

X 

04B 


FIGURE 3. Nonequivalent dependent variables design with multiple measures in repeated waves. 


the full range of classroom activity, which included the teach- 
ers’ instructional behaviors and students in their small groups. 
A videographer with a mobile camera captured close-up 
views of students working and conversing. The two cameras 
produced more than 30 1-hr tapes, which were digitized and 
compressed in MPEG-1 encoding format. They were then 
transferred to video analysis software that made it possible to 
link the classroom lessons to time points of the observers’ 
field notes. MT1 and MT2 were observed to have engaged in 
95% and 97% out of the total number of activities, respec- 
tively. Interrater reliability was 99%. 

Results 

Math Performance 

Overall Comparisons. We compared the effects of two 
EAI problems on the math skills of high-achieving students 
in a pre-algebra class, average-achieving students in typical 
math classes, and students with and without LD in an inclu- 
sive math class. Multiple comparison procedures tested dif- 
ferences on each measure by test wave and by disability status. 
An alpha level of .05 was chosen for the test of each of the 
predicted wave and achievement patterns, and Fisher’s Least 
Significant Difference (LSD) method or Holm’s sequentially 
rejective multiple comparison procedure was used (Holm, 
1979; Seaman, Levin, & Serlin, 1991) to control Type I error 
rate among the between-group comparisons. 

Table 3 shows the means and standard deviations of the 
students in the six math classes. We conducted two separate 
two-way split plot analysis of variances (ANOVAs) with re- 
peated measures on students’ scores, one for each measure. 
The within-student factor was instruction ( Kim's Komet Test 
Wave 1, 2, 3 ox Fraction of the Cost Test Wave 1, 2, 3, 4), and 
the between-students factor was class (inclusive, typical, pre- 
algebra). On the KKC, there was a main effect for instruction, 


TABLE 3. Mean and Standard Deviations of Inclusive, 
Typical, and Pre-Algebra Classes by Test Wave 


Achievement level 


Measure 


Kim’s Komet 
Challenge 3 

Fraction of the 
Cost Challenge b 

M 

SD 

M 

SD 

Test Wave 1 





Inclusive 

7.73 

5.78 

6.35 

3.76 

Typical 

8.37 

4.44 

8.95 

4.94 

Pre-Algebra 

16.42 

8.02 

13.61 

6.88 

Overall 

10.10 

6.64 

9.35 

5.58 

Test Wave 2 





Inclusive 

26.59 

5.96 

8.06 

5.61 

Typical 

23.48 

9.02 

11.36 

5.33 

Pre-Algebra 

32.88 

2.57 

17.28 

3.83 

Overall 

26.25 

8.33 

11.87 

5.83 

Test Wave 3 





Inclusive 

22.05 

7.64 

8.41 

6.44 

Typical 

12.75 

7.94 

10.95 

6.08 

Pre-Algebra 

26.77 

9.36 

17.03 

4.58 

Overall 

17.79 

10.18 

11.62 

6.45 

Test Wave 4 





Inclusive 

— 

— 

19.06 

6.79 

Typical 

— 

— 

20.55 

6.45 

Pre-Algebra 

— 

— 

26.94 

6.33 

Overall 

— 

— 

21.45 

9.35 


Note. — = not tested. 

a n = Inclusive 22, Typical 65, Pre-Algebra 26. ^ n = Inclusive 17, Typical 64, 
Pre- Algebra 18. 


F{ 2, 109= 157.88, p < .001, q 2 = .59; for class, F{2, 109) = 
41.67, p < .001, q 2 = .43; and for instruction-by-class interac- 
tion, F( 4, 109) = 5.93, p < .001, q 2 = . 10. On the FFC, there 
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All Classes 



(Weekl) (Week 4) (Week 19) (Week 24) 

Test Waves 


KKC — A — Pre-Algebra — # — Typical — H — Inclusive 

FCC ...A” Pre-Algebra •••O" Typical Inclusive 


FIGURE 4. Performances of students in the pre-algebra, typical, and inclusive 
classes. 


was a main effect for instruction F(3, 93) = 107.90,/? < .001, 
r)- = .53, for class, F( 2, 93) = 18.75, p < .001, r| 2 = .28, but 
not for instruction-by-class, F( 6, 93) = .38,/? = .89, r| 2 = .01. 
Figure 4 shows the performance of students on both measures 
at each test wave. 

Table 4 shows the main effects comparisons for the per- 
formances of students by test wave and by class. Overall, stu- 
dents scored higher on the KKC in Test Wave 2 (posttest) and 
in Test Wave 3 (maintenance) than in Test Wave 1, and stu- 
dents in the pre-algebra and inclusive classes outperformed 
students in the typical classes. On the FCC, students scored 
higher in Test Wave 4 (posttest) than in Test Wave 3 (pretest), 
but no differences were found between Test Waves 2 and 3 
(double pretest). Students in the pre-algebra class scored 
higher than students did in the typical classes who, in turn, 
scored higher than students in the inclusive class did. 

Table 5 shows the comparisons of improvement scores 
by class across assessment waves. These scores show whether 
the achievement gaps that may have existed at pretest levels 
stayed the same or increased at posttest. On the KKC, there 
were no differences between classes in the amount of im- 
provement students made between Test Waves 1 and 2. How- 
ever, students in both the pre-algebra class and the inclusive 
class showed greater improvement between Test Waves 1 and 


3 than did students in the typical classes. On the FFC, no dif- 
ferences in improvement were found between classes over test 
waves. 

Comparisons of Students With and Without LD. The 

procedures described above were also used to analyze the per- 
formance of students with and without LD in the inclusive 
class. Table 6 shows the means and standard deviations for 
each group of students at each test wave. We conducted two 
separate two-way split plot ANOVAs with repeated measures 
on students’ scores, one for each measure. The within-student 
factor was instruction (Test Waves 1, 2, 3 for Kim’s Komet and 
Test Waves 1,2, 3, 4 for Fraction of the Cost), and the be- 
tween-students factor was disability status. On the KKC, there 
was a main effect for instruction, F( 2, 20) = 76.36, /? < .001, 
r| 2 = .79, and for instruction-by-disability interaction, F(2, 20) 
= 4.01,/? = .03, r| 2 = .17, but not for disability, F( 1 , 20) = 1.26, 
p = .28, vp = .06. On the FFC, there was a main effect for in- 
struction, F( 3, 14) = 30.68, /? < .001, r| 2 = .67, and for dis- 
ability, F( 1, 14) = 6.52, /? = .02, r\~ = .30, but not for 
instruction-by-disability interaction, 7^(3, 14) = 1.40,/? = .26, 
T] 2 = .09. Figure 5 shows the performance of students with LD 
and students without LD on the KKC and FCC for each test 
wave. 
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TABLE 4. Comparisons on Kim’s Komet Challenge (KKC) and Fraction of the Cost 
Challenge (FCC) by Test Wave and Class 


Measure and comparison 

Contrast estimate 

SE 

Observed t 

p value 

Kim's Komet Challenge 
Test wave comparisons 

Wave 2 vs. Wave 1 

16.15 

0.76 

21.35 

p < .001* 

Wave 3 vs. Wave 1 

7.69 

0.89 

8.64 

p < .001* 

Achievement comparisons 

Pre-algebra vs. Typical 

10.49 

1.15 

9.12 

p < .001* 

Pre-algebra vs. Inclusive 

6.57 

1.44 

4.56 

p < .001* 

Typical vs. Inclusive 

-3.92 

1.23 

-3.19 

p = .002* 

Fraction of the Cost Challenge 
Test wave comparisons 

Wave 2 vs. Wave 1 

2.52 

0.53 

4.76 

p < .001* 

Wave 3 vs. Wave 1 

2.27 

0.61 

3.74 

p = .001* 

Wave 3 vs. Wave 2 

-0.25 

0.50 

-0.49 

p = .624 

Wave 4 vs. Wave 3 

9.83 

0.68 

14.55 

p < .001* 

Achievement comparisons 

Pre-algebra vs. Typical 

5.76 

1.12 

5.14 

p < .001* 

Pre-algebra vs. Inclusive 

8.24 

1.42 

5.80 

p < .001* 

Typical vs. Inclusive 

2.48 

1.15 

2.16 

p = .03* 

*Significant, a = .05 familywise, Fisher’s 

TABLE 5. Comparisons of 

LSD. ** Significant, a = .05 familywise, Holm. 

Improvement Scores for Inclusive, Typical, and Pre-Algebra 

Classes by Test Wave 

Measure and comparison 

Contrast estimate 

SE 

Observed t 

p value 

Kim's Komet Challenge 
Wave 2 and Wave 1 

Pre-algebra vs. Typical 

1.35 

1.85 

0.73 

p = .466 

Pre-algebra vs. Inclusive 

-2.40 

2.31 

-1.04 

p = .301 

Typical vs. Inclusive 

-3.76 

1.97 

-1.91 

p = .059 

Wave 3 and Wave 1 

Pre-algebra vs. Typical 

5.96 

2.00 

2.98 

p = .004** 

Pre-algebra vs. Inclusive 

-3.97 

2.50 

-1.59 

p= .115 

Typical vs. Inclusive 

-9.93 

2.13 

-4.66 

p < .001** 

Fraction of the Cost Challenge 
Wave 1 and Wave 2 

Pre-algebra vs. Typical 

1.26 

1.41 

0.89 

p = .373 

Pre-algebra vs. Inclusive 

1.96 

1.78 

1.10 

p = .274 

Typical vs. Inclusive 

0.70 

1.44 

0.49 

p = .628 

Wave 1 and Wave 3 

Pre-algebra vs. Typical 

1.42 

1.62 

0.01 

p = .383 

Pre-algebra vs. Inclusive 

1.36 

2.05 

0.66 

p = .510 

Typical vs. Inclusive 

-0.06 

1.65 

-0.04 

p = .972 

Wave 3 and Wave 4 

Pre-algebra vs. Typical 

0.32 

1.81 

0.18 

p = .859 

Pre-algebra vs. Inclusive 

-0.73 

2.29 

-0.32 

p = .751 

Typical vs. Inclusive 

-1.05 

1.85 

-0.01 

p = .571 


**Significant, a = .05 family wise, Holm. 
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TABLE 6. Means and Standard Deviations of Students in 
the Inclusive Class With LD and Without LD by Test Wave 




Measure 



Kim’s Komet 
Challenge 

Fraction of the 
Cost Challenge 

Disability 

M 

SD 

M 

SD 

Test Wave 1 

LD 

5.17 

2.48 

5.67 

4.06 

Non-LD 

10.80 

7.16 

7.12 

3.48 

Overall mean 

7.73 

5.78 

6.35 

3.76 

Test Wave 2 

LD 

24.83 

5.24 

5.67 

5.94 

Non-LD 

28.70 

6.34 

10.75 

3.99 

Overall mean 

26.59 

5.96 

8.06 

5.61 

Test Wave 3 

LD 

23.33 

6.14 

4.89 

4.20 

Non-LD 

20.50 

9.22 

12.37 

6.39 

Overall mean 

22.05 

7.64 

8.41 

6.44 

Test Wave 4 

LD 

— 

— 

16.89 

5.63 

non-LD 
Overall mean 



21.50 

19.06 

7.50 

6.79 


Note. LD = learning disabled; non-LD = non-learning disabled; — = not tested. 


Table 7 shows comparisons of achievement and improve- 
ment scores of students in the inclusive class by disability sta- 
tus across test waves. On the KKC, students scored higher in 
Test Wave 2 (posttest) and Test Wave 3 (maintenance) than in 
Test Wave 1 (pretest). Improvement from Test Wave 1 to 3 (pre- 
test to maintenance) was larger for students with LD than it 
was for students without LD. On the FCC, students scored 
higher in Test Wave 4 (posttest) than in Test Wave 3 (pretest), 
but there were no differences in achievement across Test 
Waves 1, 2, and 3. Improvement scores between Test Waves 3 
and 4 showed no difference between students with LD and 
students without LD. 

Item-/Concept-Level Analyses. Tables 8 and 9 indicate 
the percentage of students who earned either full or partial 
credit on items within each concept assessed on KKC and 
FCC, respectively. On the KKC, the difference in the propor- 
tion of students earning credit before and after instruction was 
greatest for calculating rates, completing data tables, under- 
standing concepts of constant speed and acceleration, and con- 
structing a graph (see Figure 1, Items 5-7). Almost all of the 
students in the pre-algebra class (92%) were able to do Item 
5 on the pretest, compared to only about half (55%) of the stu- 
dents with LD. However, post-instruction results showed that 
almost all of the students with LD (91%) received at least par- 
tial credit for their answers, and 68% of them earned a per- 


Inclusive Class 



(Weekl) (Week 4) (Week 19) (Week 24) 

Test Waves 


KKC 


Students with LD 

— H — Students without LD 

FCC 

" 0 ” 

Students with LD 

• Students without LD 


FIGURE 5. Performances of students with and without LD in the inclusive class. 
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TABLE 7. Achievement and Improvement Scores of Students in the Inclusive Class With 
LD and Without LD by Test Wave 


Measure and comparison 

Contrast estimate 

SE 

Observed t 

p value 

Kim's Komet Challenge 
Achievement 

Wave 2 vs. Wave 1 

18.86 

1.23 

15.36 

p < .001* 

Wave 3 vs. Wave 1 

14.32 

1.90 

7.55 

p < .001* 

Improvement score 
non-LD vs. LD 
Wave 2 vs. Wave 1 

-1.77 

2.50 

-0.71 

p = .487 

Wave 3 vs. Wave 1 

-8.47 

3.41 

-2.48 

p = .022* 

Fraction of the Cost Challenge 
Achievement 

Wave 2 vs. Wave 1 

1.71 

1.07 

1.60 

p = .129 

Wave 3 vs. Wave 1 

2.06 

1.26 

1.64 

p = .121 

Wave 4 vs. Wave 3 

10.65 

1.49 

7.17 

p < .001** 

Improvement score 
non-LD vs. LD 
Wave 2 vs. Wave 1 

3.63 

2.00 

1.82 

p = .09 

Wave 3 vs. Wave 1 

6.03 

2.08 

2.90 

p = .011** 

Wave 4 vs. Wave 3 

-2.88 

2.98 

-0.97 

p = .350 


Note. Non-LD = non-learning disabled; LD = learning disabled. 

*Significant, a = .05 familywise, Fisher’s LSD. **Significant, a = .05 familywise, Holm. 


feet score. On Item 6, none of the students with LD earned 
credit prior to instruction, whereas all of them received at least 
partial credit, and 94% earned full credit after instruction. In 
fact, a greater percentage of students with LD understood this 
item than did the students without LD who were in the same 
classroom. However, Item 7 showed that students with LD 
were much less successful in constructing a graph to repre- 
sent their data. 

On the FCC posttest (Test Wave 4), similar patterns 
emerged. All the students were at least partially successful in 
computing the amount of money each friend would contribute 
to buy materials for the skateboard ramp. Students with LD 
learned the most from instruction that focused on developing 
a materials list from a schematic plan, reading a tape measure 
to Vn", converting feet and inches to inches alone, and com- 
paring lengths when they are provided as mixed numbers 
(Item 3). However, most of the students in all ability groups 
had considerable difficulty when it came to showing the most 
economical way of cutting the lengths of wood (Item 4). If 
students were unable to figure out this item, they could not 
compute the cost of the project correctly. 


Discussion 

The overall goal of this study was to measure the effective- 
ness of a reform-oriented instructional method called EAI on 


the achievement of middle school students. Specific objec- 
tives of the research were twofold. First, we tested the effects 
of EAI on the problem-solving performances of students with- 
out disabilities who were average and high achievers. Then, 
using their scores as performance benchmarks, we measured 
the treatment effects of EAI on students with LD. 

Summary of Findings 

Results indicated that students in all six classes, including stu- 
dents with LD, benefited from both sets of EAI problems. On 
the FCC, which assessed linear measurement, computing frac- 
tions, and reading three-dimensional schematic plans, the ob- 
tained results followed predicted patterns. That is, students in 
the pre-algebra class scored higher than did students in both 
the typical and inclusive classes, and students in the typical 
classes scored higher than did the combination of students 
with and without LD in the inclusive class. However, there 
were no differences between groups in the size of improve- 
ment. As for the students with LD, they scored lower than their 
classmates without LD did, but there was no difference be- 
tween the groups in improvement. 

A somewhat different pattern of results was found for 
KKC. As on the FCC, the pre-algebra students outscored stu- 
dents in the typical and inclusive classes, but students in the 
inclusive class scored higher than students in the typical 
classes did. Furthermore, scores at follow-up 14 weeks after 
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TABLE 8. Percentage of Students Earning Partial or Full Credit on the Kim’s Komet Challenge (KKC) 


Class/ achievement 


Concepts measured 

Test wave 

Pre-algebra 

Typical 

Inclusive 

non-LD 

Inclusive 

LD 

Total 

Single/Two-step word problems 

1 

100 

83 

100 

82 

91 

d = r x t 
r = d/t 

2 

100 

94 

100 

73 

92 

(2 points) 

Difference 

0 

+11 

0 

-9 

+1 

Interpret line graphs, 1 and 2 data 

1 

100 

100 

91 

91 

95 

lines, distance vs. time 

2 

100 

100 

91 

100 

98 

(4 points) 

Difference 

0 

0 

0 

+9 

+3 

Interpret data tables, compute 

1 

92 

74 

73 

55 

73 

and compare rates (Item #5) 

2 

100 

97 

100 

91 

97 

(16 points) 

Difference 

+8 

+23 

+27 

+36 

+24 

Interpret figures, compute rates, 

1 

8 

6 

9 

0 

6 

understand concepts of constant 
speed and acceleration (Item 6) 

2 

100 

62 

91 

100 

88 

(6 points) 

Difference 

+92 

+56 

+82 

+100 

+82 

Construct graph, scale data, label 

1 

65 

36 

36 

9 

37 

x and y axes, plot rates, draw line 
of best fit (depends on previous item) 
(Item #7) 

2 

100 

80 

91 

55 

81 

(6 points) 

Difference 

+35 

+44 

+55 

+46 

+44 

Predict rates (answer depends on 

1 

8 

2 

0 

0 

2 

previous item) 

2 

54 

41 

45 

9 

37 

(2 points) 

Difference 

+46 

+39 

+45 

+9 

+35 


Note. Non-LD = non-learning disabled; LD = learning disabled. 


EAI showed that students in the inclusive class widened their 
performance margin and did not lose ground to the high- 
achieving students in the pre-algebra class. Comparing just 
the students in the inclusive class, students with LD scored 
slightly lower than did their classmates at both pretest and 
posttest, but their improvement from pretest to maintenance 
test was larger than that of their classmates without LD. 

Item analyses also showed that students at all perfor- 
mance levels learned important skills and concepts, such as 
interpreting data tables and line graphs, calculating and com- 
paring rates, interpreting schematic plans, and working with 
and converting linear measures. However, close inspection of 
these findings also revealed that many of the students, even 
those who were considered to be high achieving, had diffi- 
culty integrating several data sources into a descriptive repre- 
sentation of the overall problem solution. The Fraction of the 
Cost problem seemed especially difficult because the solution 
depended on understanding and working with fractions, a 


commonly reported area of misunderstanding for many stu- 
dents (Empson, 2003; Mack, 1995; Saxe, Taylor, McIntosh, 
& Gearhart, 2005). 

Theoretical and Practical Implications 

Deshler et al. (2001) suggested that the achievement gap of 
students with LD in secondary inclusive classes could be re- 
duced by (1) developing curricular materials that are appro- 
priate for diverse groups of students and (2) training teachers 
to use it in ways students understand. On the surface, these 
goals appear to be common sense, but this study and previous 
EAI research have demonstrated that the path toward achiev- 
ing these goals is neither easy nor straightforward. Although 
our findings suggest that EAI can address some of the learn- 
ing needs of diverse groups of students, including students with 
LD, they also reveal the complexity involved in integrating 
cognitive supports in both the learning materials (e.g., multi- 
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TABLE 9. Percentage of Students Earning Partial 

or Full Credit 

on the Fraction of the Cost Challenge 

(FCC) 





Class/ achievement 



Concepts measured 

Test wave 

Pre-algebra 

Typical 

Inclusive 

non-LD 

Inclusive 

LD 

Total 

Computing money each student 

3 

100 

100 

100 

100 

100 

contributed (Compute money from 
bank statement, compute percent) 

4 

100 

100 

100 

100 

100 

(3 points) 

Difference 

0 

0 

0 

0 

0 

Interpret schematic plans (to 1/4 inch). 

3 

89 

64 

100 

56 

87 

complete materials list, convert feet and 
inches to inches. 

4 

100 

97 

88 

89 

92 

(12 points) 

Difference 

+11 

+33 

-12 

+33 

+5 

Read tape measure to 1/8 inch, com- 

3 

100 

90 

100 

56 

90 

plete data table (quantity, description, 
length in feet, length in inches) (Item #3) 

4 

95 

97 

88 

89 

95 

(4 points) 

Difference 

-5 

+7 

-12 

+33 

+5 

Compare and label lengths of wood 

3 

72 

28 

50 

44 

49 

(depends on constructing data table 
in previous item) 

4 

94 

80 

75 

56 

76 

(1 point) 

Difference 

+22 

+52 

+25 

+ 12 

+27 

Show most economical combinations, 

3 

22 

0 

13 

0 

9 

add and subtract mixed numbers, 
interpret figures (depends on answer 
in previous two items) 

4 

89 

53 

50 

44 

59 

(5 points) 

Difference 

+67 

+53 

+37 

+44 

+50 

Compute/Label combinations of wood. 

3 

17 

3 

0 

0 

5 

add and subtract mixed numbers, interpret 
tables and figures (depends on answers 
to previous items) (Item #4) 

4 

78 

42 

38 

22 

45 

(6 points) 

Difference 

+61 

+39 

+38 

+22 

+40 

Computing total cost, computing 

3 

0 

0 

0 

0 

0 

subtotals for materials (screws, dimen- 
sion lumber), calculating sales tax (correct 
answer depends on correct answers to 
previous four items) 

4 

44 

17 

25 

0 

22 

(6 points) 

Difference 

+44 

+ 17 

+25 

0 

+22 


Note. Non-LD = non-learning disabled; LD = learning disabled. 


media tools) and the instructional methods (e.g., managing 
problem-solving groups). 

The need for such supports is based on cognitive load 
theory (CLT; Chandler & Sweller, 1991; Sweller, 1988), which, 
in its simplest form, suggests that all learners have limited 
working memory and that learning tasks should be structured 
in ways that do not overload it. One important consideration 
relates to what is called element interactivity, the extent to 
which relevant elements of content embedded in curricular ma- 
terials interact. High interactivity materials cannot be fully un- 


derstood until all the elements are processed simultaneously. 
Adding story contexts, visual representations, and multimedia 
applications to materials at this level of complexity can be ex- 
tremely helpful in reducing cognitive overload (Mayer & Mor- 
eno, 2003; Mousavi, Low, & Sweller, 1995; Rittle-Johnson & 
Koedinger, 2005; Tabbers, Martens, & van Merrienboer, 2004). 

The EAI problems in this study and those frequently 
encountered in life outside of school would most likely be sit- 
uated high on the interactivity continuum. In some of our pre- 
vious research, the sole use of the video-based anchor, in the 
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absence of additional opportunities afforded by the learning 
tools and hands-on applications, did not meet the learning 
needs of some low-achieving students. Recent updates to Frac- 
tion of the Cost include several cognitive supports, among these 
a virtual tape measure, an interactive project in which students 
build and fly a hovercraft, and an explicit instruction module 
for helping students understand how to compute fractions. All 
of these were necessary additions to the EAI curriculum. 

In addition to the multimedia tools, many students needed 
to “see” the problems (i.e., hovercraft, car pentathlon) in real 
life. In poststudy conversations, the math teachers thought the 
car pentathlon competition was the most effective tool of all. 
Students used their smart tool, or graph, to help them predict 
the correct height from which to release their cars to navigate 
the stunts at the end of the straightaway. Commenting on the 
performance of his students in his inclusive class, the math 
teacher pointed out, “the people who won [the pentathlon 
event] are normally people who are not successful in math. 
The people in my classes — first, second, and third — were peo- 
ple who never got the highest grades and had more difficul- 
ties, even the kids in my algebra class. The girl who won it 
normally comes on a daily basis and has a lot wrong on her 
homework. But she used her smart tools.” 

The teachers also had to be well prepared to deal with 
the complexity of the materials and the needs of their students. 
Recent studies have shown that content knowledge and the 
skills used to teach the math content are important contribu- 
tors to student achievement in general education (e.g.. Hill, 
Rowan, & Ball, 2005). The teachers had to anticipate the 
“bugs” in their students’ problem-solving procedures to know 
when and how to redirect their efforts. Second, they had to 
know where to find the learning tools in the software and how 
to operate the other technology tools, such as the infrared 
timer used in the pentathlon. Finally, the math and special ed- 
ucation teachers had to know how to facilitate the work of stu- 
dents in small groups so that all students had opportunities for 
giving their input, an especially important skill in the inclu- 
sive classroom. 

The relative lack of training and experience of one of the 
math teachers may explain why the scores of typical achiev- 
ers were lower on KKC than those of students in the inclusive 
class on the posttest and maintenance test waves. It was the 
first time this teacher had taught Kim’s Komet and it is possi- 
ble, if not probable, that she was unfamiliar with all the nu- 
ances involved in teaching and learning the EAI problems. 
Although observational data showed that she was an excellent 
teacher, unfamiliarity with the technologies and the subtleties 
of the problems may have led to her students’ lower scores. 

Finally, we comment on the complexity of the research 
endeavor itself as it relates to intervention research. As spe- 
cial educators have pointed out (e.g., Gersten, 2005; Graham, 
2005; Palincsar, 2005), this type of research takes consider- 
able time, from the theoretical justification for developing the 
new or modified intervention, to development and pilot work, 
to research design and implementation, and finally to the final 


analyses and reporting. As more students with disabilities are 
included in general education classrooms, developing and im- 
plementing curricula that meets the needs of students with and 
without disabilities will continue to present a formidable chal- 
lenge, especially at the secondary level (Conderman & Kat- 
siyannis, 2002; Fuchs & Fuchs, 1998). 

Limitations 

These findings suggest that EAI can have important effects on 
learning, especially for students with LD, but we temper our 
results with the following limitations. First, the research de- 
sign had advantages, as we noted earlier, but it also had sev- 
eral disadvantages. Probably most important, it did not assess 
the efficacy of the EAI approach versus other forms of an- 
chored instruction or typical instruction, although compari- 
sons have been made in previous work (e.g., Bottge, 1999; 
Bottge, Heinrichs, Chan, & Serlin, 2001). The design also re- 
quired students to be tested several times on concepts they had 
not yet learned. Double pretests and repeated measurements 
helped minimize threats to internal validity (e.g., history), but 
teachers had to attend closely to student morale at all test 
waves. 

Second, the outcome measures were aligned closely to 
situations taught during EAI, and thus we do not know to what 
extent students could generalize their understandings to new 
contexts. On the other hand, the pencil-paper format of the 
tests may have limited students’ ability to show all they had 
learned. Judging from the scores of even the highest achiev- 
ers, the tests were difficult. One reason for this may be a mis- 
match between learning and testing formats. Although the 
concepts tested were aligned with the instructional content, 
the paper tests did not offer students possibilities for search- 
ing problem contexts in the same way as during instruction. 
Creating interesting and meaningful problem contexts in 
paper format may have important implications for all students, 
especially for those who find it particularly difficult to picture 
text-based problems. 

Third, although all of the seventh-grade students with 
LD were included in the study, the number was quite small 
and thus may have contributed to relatively low power. We 
frequently encounter this difficulty when conducting inter- 
vention studies in which there are few students with disabili- 
ties in each inclusive classroom. For this reason, we have tried 
to provide adequate detail of the interventions, measures, and 
results. 

Conclusion 

Ceci and Roazzi (1994) made the insightful observation, “we 
cannot conclude that children lack certain cognitive abili- 
ties just because they do not exhibit them in a given context” 
(p. 93). We are cautiously optimistic about the possibilities 
technology-assisted, well-structured learning environments 
can have in reducing the differences between students with di- 
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verse skill levels. A decade ago, Darling-Hammond (1996) 
suggested educators should develop teaching methods that go 
far beyond dispensing information, focus on challenging cur- 
riculum goals for all students (i.e., be learning centered ), and 
meet the needs and interests of individual learners (i.e., be 
learner centered). In this study, we have attempted to realize 
these goals by providing constructed contexts in which stu- 
dents at all achievement levels could succeed. 
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